First version drafted (https://gitlab.com/ska-telescope/src/ska-rucio-extended-client)
Created recursive tree functions to parse the structure, just need to implement download.
Fixed bug in list_dids that was exhausting database connections.
Found argocd jobs aren't syncing.
Tested upload, seems to work. Started download model. Bits left to do:
Note that hierarchical_tests:test50* has a long lifetime.
Some issues with number of connections to sqla, changed server config:
pool_recycle = 60
pool_size = 30
pool_max_overflow = 10
schema = testSeems to have "fixed it" (not really, still issues).
Up to the point where i'm trying to add files from datasets in to the relationships list of tuples.
This can then be parsed with treelib, e.g. by looking first for the relationship with no parent (i.e. the root), then looking for relationships where this root is the parent. This can be done recursively to get the full directory structure.
Finished initial upload model, uploaded to repo.
Nearly at the point of attempting to upload a folder.
Spoke to Jan David (astron) about dashboards in the morning.
Fixed Rucio logs not coming through (restarted conveyor).
Moved on to looking at hierarchical datasets in Rucio.
Planning week for PI16.
Mostly refactoring rucio-task-manager and making the slack bot work again.
Some time spent creating the IVOA ObsCore prototype.